Neural network training and validation rely on the availability of large high-quality datasets. However, in many cases only incomplete datasets are available, particularly in health care applications, where each patient typically undergoes different clinical procedures or can drop out of a study. Since the data to train the neural networks need to be complete, most studies discard the incomplete datapoints, which reduces the size of the training data, or impute the missing features, which can lead to artefacts. Alas, both approaches are inadequate when a large portion of the data is missing. Here, we introduce GapNet, an alternative deep-learning training approach that can use highly incomplete datasets. First, the dataset is split into subsets of samples containing all values for a certain cluster of features. Then, these subsets are used to train individual neural networks. Finally, this ensemble of neural networks is combined into a single neural network whose training is fine-tuned using all complete datapoints. Using two highly incomplete real-world medical datasets, we show that GapNet improves the identification of patients with underlying Alzheimer's disease pathology and of patients at risk of hospitalization due to Covid-19. By distilling the information available in incomplete datasets without having to reduce their size or to impute missing values, GapNet will permit to extract valuable information from a wide range of datasets, benefiting diverse fields from medicine to engineering.

Think back to the first time you heard the phrase "neural networks" or "neural nets" -- perhaps it's right now -- and try to remember what your first impression was. As an Applied Math and Economics major with a newfound interest in data science and machine learning, I remember thinking that whatever neural networks are, they must be extremely important, really cool, and very complicated. I also remember thinking that a true understanding of neural networks must be on the other side of a thick wall of prerequisite knowledge including neuroscience and graduate mathematics. Through taking a machine learning course with Professor Samuel Watson at Brown, I have learned that three of the previous four statements are true in most cases -- neural nets are extremely important, really cool, and they can be very complicated depending on the architecture of the model. But most importantly, I learned that understanding neural networks requires minimal prerequisite knowledge as long as the information is presented in a logical and digestable way.

The first time I came across RNNs, I was completely baffled. How can a network even remember things? Recurrent Neural Networks have proved to be effective and popular for processing sequential data ever since the first time they emerged in the late 1980s. Recurrent Neural Networks have been derived from vanilla Feed Forward Neural Networks. They have so-called memory elements that help the network remember previous outputs. They are Recurrent because they repeatedly perform the same task for every element in the sequence, with the output being dependent on the previous computations. Recurrent Neural Networks (RNNs) have been a huge improvement over the vanilla neural network. A typical vanilla neural network calculates an output on the current input and weights with a limitation of predetermined fixed input size. In this article, we will go over the architecture of RNNs, with just enough math by taking the example of Elman Network.

This tutorial was originally posted here on Ben's blog, GormAnalysis. Artificial Neural Networks are all the rage. One has to wonder if the catchy name played a role in the model's own marketing and adoption. I've seen business managers giddy to mention that their products use "Artificial Neural Networks" and "Deep Learning". Would they be so giddy to say their products use "Connected Circles Models" or "Fail and Be Penalized Machines"? But make no mistake – Artificial Neural Networks are the real deal as evident by their success in a number of applications like image recognition, natural language processing, automated trading, and autonomous cars.

I am somewhat new to this deep learning thing. I fell under the spell when I realized that a computation of a layer of a multilayer perceptron ("vanilla neural network") is simply a composition of a linear map and the the vectorized activation function. Now, convolutional neural networks are seemingly something more complex. But then I realized, aren't convolutional neural networks just a special case of this "vanilla neural network"? In convolutional neural networks, each new feature is computed out of just some strict subset of features from the previous layer. Only some features are "selected" to the weighted sum.